Search for: All records

This paper presents approaches to mean-field control, motivated by distributed control of multi-agent systems. Control solutions are based on a convex optimization problem, whose domain is a convex set of probability mass functions (pmfs). The main contributions follow: 1. Kullback-Leibler-Quadratic (KLQ) optimal control is a special case, in which the objective function is composed of a control cost in the form of Kullback-Leibler divergence between a candidate pmf and the nominal, plus a quadratic cost on the sequence of marginals. Theory in this paper extends prior work on deterministic control systems, establishing that the optimal solution is an exponential tilting of the nominal pmf. Transform techniques are introduced to reduce complexity of the KLQ solution, motivated by the need to consider time horizons that are much longer than the inter-sampling times required for reliable control. 2. Infinite-horizon KLQ leads to a state feedback control solution with attractive properties. It can be expressed as either state feedback, in which the state is the sequence of marginal pmfs, or an open loop solution is obtained that is more easily computed. 3. Numerical experiments are surveyed in an application of distributed control of residential loads to provide grid services, similar to utility-scale battery storage. The results show that KLQ optimal control enables the aggregate power consumption of a collection of flexible loads to track a time-varying reference signal, while simultaneously ensuring each individual load satisfies its own quality of service constraints. 

There is enormous flexibility potential in the power consumption of the majority of electric loads. This flexibility can be harnessed to obtain services for managing the grid: with carefully designed decision rules in place, power consumption for the population of loads can be ramped up and down, just like charging and discharging a battery, without any significant impact to consumers' needs. The concept is called Demand Dispatch, and the grid resource obtained from this design virtual energy storage (VES). In order to deploy VES, a balancing authority is faced with two challenges: 1. how to design local decision rules for each load given the target aggregate power consumption (distributed control problem), and 2. how to coordinate a portfolio of resources to maintain grid balance, given a forecast of net-load (resource allocation problem).Rather than separating resource allocation and distributed control, in this paper the two problems are solved simultaneously using a single convex program. The joint optimization model is cast as a finite-horizon optimal control problem in a mean-field setting, based on the new KLQ optimal control approach proposed recently by the authors.The simplicity of the proposed control architecture is remarkable: With a large portfolio of heterogeneous flexible resources, including loads such as residential water heaters, commercial water heaters, irrigation, and utility-scale batteries, the control architecture leads to a single scalar control signal broadcast to every resource in the domain of the balancing authority. Keywords: Smart grids, demand dispatch, distributed control, controlled Markov chains. 

A new stochastic control methodology is introduced for distributed control, motivated by the goal of creating virtual energy storage from flexible electric loads, i.e. Demand Dispatch. In recent work, the authors have introduced Kullback- Leibler-Quadratic (KLQ) optimal control as a stochastic control methodology for Markovian models. This paper develops KLQ theory and demonstrates its applicability to demand dispatch. In one formulation of the design, the grid balancing authority simply broadcasts the desired tracking signal, and the hetero-geneous population of loads ramps power consumption up and down to accurately track the signal. Analysis of the Lagrangian dual of the KLQ optimization problem leads to a menu of solution options, and expressions of the gradient and Hessian suitable for Monte-Carlo-based optimization. Numerical results illustrate these theoretical results.